Test coverage – the percentage of code exercised by your tests – is a necessary but not sufficient condition for complete testing of your code. Test coverage only shows whether the lines of code have been exercised, not how. (Is your testing code a Tai Chi instructor, or are your tests putting your code through an ironman triathlon?)
One way that over-reliance on test coverage can betray you is that test coverage does not cover unwritten code, like the additional necessary name validation in a system under conversion from ASCII to Unicode, or the handling of numeric exceptions in a bignum division routine. Another example is the combinatorial condition testing problem – test code that examines all conditions in an IF, but does not examine all possible combinations of those conditions will show 100% test coverage for that IF test but does not cover all condition combinations (and therefore code paths).
Axioms – the assumptions about the code (and system) under test – are the foundation of what keeps test code with 100% coverage from completely testing the system. Test code usually assumes that the CPU will correctly perform its computations and logic operations. Test code oftentimes assumes that the database will return records in the database and not return records not in the database (flaky hardware does not go well together with standard commercial databases). There are numerous other assumptions about the code under test that are usually made in test code (configuration files won't be rewritten in the middle of reading them, not all combinations of IF conditions need be tested, etc.)
Test code will always need some assumptions about the system used for the test (Godel's Incompleteness Theorem guarantees this). Handling the challenge of choosing assumptions for your test code is a matter of prioritizing which assumptions are least likely to break on the test system. Testing a new, buggy processor requires a different set of assumptions than testing glue code between an electronic CAD system, Oracle, and a mainframe ERP package. You aren't testing all of your code if you don't have 100% test coverage, but you may not be testing all of your code (or code paths, at least) even when you have 100% test coverage.
Let's get this out of the way, testing is good, testing is wonderful, test, test, test.
But, don't forget:
"Testing can never show the absence of errors, only that there are errors." -- Edgar Dykstra
